Scene Direction Based Reference In Drama Scenes

Our research target is reference relations between descriptions of script and an actor/actress who actually plays in the drama scene that correspond to the scene direction which is a part of the script. In this paper, first we analyze sentences used as the scene directions, and classify them. Then we propose the rules to extract subjects and predicates from those sentences. With the extracted subjects and predicates, we build the existence/action map that explains the situations happening on each scene. The existence/action map we build describes scenes very correctly as for whether each player appears in each scene or not. Our experiment shows that the recall is around 80% and the precision is over 90%. This means that our system of inferring reference relations works well for scene directions. Then we develop the scene retrieval system in which this map is used to retrieve scenes from the drama video database according to the input query. We also show some experimental results of our retrieval system. 1 I n t r o d u c t i o n In conventional multimedia retrieval systems, image data are indexed manually because image processing technologies have not yet provided us with image understanding methods that are powerful enough to automatically extract useful indices from image data themselves. Then, in order to enhance multimedia retrieval systems, we have to employ other types of information source. Tha t is why we focus here on drama scripts that are apparently natural language media. However it is obvious that only using scripts does not guarantee effective retrieval. The essential point is, of course, the combination of information from language resource and image data. This combination can drastically improve the quality and efficiency of multimedia retrieval. From this point of view, reference relations between a description of script and an actor/actress who actually plays in the drama scenes that correspond to the script are the most useful pieces of information for scene retrieval, and consequently they become our research targets. If we successfully identify this kind of reference, it is a great help for multimedia information retrieval, especially scene retrieval from data base of drama videos. However the whole script of drama is so complicated that we cannot deal with by today's natural language processing technologies. Then we concentrate our focus here on s cen e d i r e c t i o n s that explain the situation of each scene in the drama, for instance, actors/actresses' position, movement, and